Drowsiness Detection System

ABSTRACT

A machine-implemented method for automated detection of drowsiness, which includes receiving from an imaging device directed at the face of an operator a series of images of the face of the operator onto processing hardware, on the processor detecting facial landmarks of the operator from the series of images to determine the level of talking by the operator, the level of yawning of the operator, the PERCLOS of the operator, on the processor detecting the facial pose of the operator from the series of images to determine the level of gaze fixation by the operator, on the processor calculating the level of drowsiness of the operator by ensembling the level of talking by the operator, the level of yawning of the operator, the PERCLOS of the operator and the level of gaze fixation by the operator, and generating an alarm when the calculated level of drowsiness of the operator exceeds a predefined value.

CROSS-REFERENCE TO RELATED APPLICATION INFORMATION

The present application is a U.S. national stage patent application,pursuant to 35 U.S.C. § 371, of PCT International Application No.:PCT/IB2019/058990, filed Oct. 22, 2019, published as WO2020/084469A1,which claims priority to U.S. provisional patent application No.62/748,596, filed Oct. 22, 2018, the contents of all of which are herebyincorporated by reference in their entirety.

TECHNICAL FIELD

This invention relates to a drowsiness detection system and amachine-implemented method for automated detection of drowsiness.

BACKGROUND

According to the Australian Transport Council, drowsy driving causesbetween 20% and 30% of all deaths and injuries on the road [1]. Studieshave shown that driving without sleep for 17 hours is comparable todriving with a blood alcohol concentration (BAC) of 0.05 (the legallimit of most countries including the USA, England and South Africaranges between 0.05% and 0.08% [2]) and driving without sleep for 21hours is comparable to a BAC of 0.15 (far over the legal limit) [1].Drowsiness also affects many other industries where vigilance isrequired and can even be problematic for office workers.

In order to solve this problem, accurate drowsiness detection is needed.This would allow systems to be developed that alert operators whendrowsiness is detected. Furthermore, it would allow companies andgovernments to monitor their operators and enforce vigilance.

Drowsiness detection has received an abundance of attention in recentyears from both automotive companies and universities. A variety ofapproaches and methods have been researched and tested. These methodscan be classified into three groups, namely, physiological methods,vehicle movement methods and computer vision methods.

Early methods made use of physiological measures such as respirationrate, heart rate and brain activity [3]. These methods are, however,intrusive as measurement devices must be attached to the operator. Thiscan be dangerous if the measurement devices restrict the movement of theoperator. Furthermore, the devices often hinder the comfort of operatorsand cause distractions, increasing the risk of an incident or accident.

Another method of drowsiness detection, geared towards drivers, is themonitoring of steering wheel movement [4] and vehicle movement. Sensorsare typically placed inside the steering wheel or dashboard of a vehicleand measure the angular velocity and acceleration. This data is thenused to classify erratic driving and swerving that is a characteristicof drowsy driving. A major drawback of this method is the large impactthat road conditions and vehicle speed have on it. If road conditionsare bad, drivers may swerve to avoid potholes and other obstacles whichcould be misinterpreted as drowsy driving. Additionally, the nature ofthe recorded data requires an extended period of analysis before anaccurate classification of drowsiness can be made. This reduces theeffectivity of the system as an accident may occur beforeclassification. Placement of the sensors may also be problematic as theymay interfere with the driver if they are not built into the vehicle.

Advances in computer vision have allowed drowsiness detection to beperformed by observing facial features such as the eyes and mouth.Facial characteristics of drowsiness include a smaller degree of eyelidopening, slow eyelid movement, increased number of blinks, yawning,nodding and gazing. Using an imaging device pointing towards theoperator, machine learning and computer vision can be used to detectthese characteristics and perform drowsiness detection. This methodbenefits from the fact that it is non-intrusive and is not affected byroad conditions or driving speed when it is performed on drivers.

In one study [5], a system was developed that monitors drowsinessthrough yawn detection. This method proved successful but resulted infalse alarms in cases where operators were tired but not drowsy. Anotherproblem was that operators often covered their mouths while yawning,obstructing the view of the camera. Another study [6] was performedwhere multiple features were calculated from the faces of drowsyoperators in order to determine which features result in the bestdrowsiness classification. The accuracy of the calculated features isshown below. It is clear that fixed gaze and percentage of eye closureover time (PERCLOS) are the best features for drowsiness detection withan accuracy of 95.6% and 93.1% respectively.

-   -   Fixed Gaze: Accuracy: 95.6%    -   PERCLOS: Accuracy: 93.1%    -   Eye Closure Duration: Accuracy: 84.4%    -   Blink Frequency: Accuracy: 80%    -   Nodding Frequency: Accuracy: 72.5%

The inventor identified a need to integrate and improve the methods andsystems to detect drowsiness with improved accuracy and to takeimmediate action upon detecting drowsiness of an operator thereby toreduce accidents related to drowsiness of an operator.

RELATED PUBLICATIONS

-   Real time driver drowsiness detection using a    logistic-regression-based machine learning algorithm (Babaeian,    Bhardwaj, Esquivel, & Mozumdar, 2016) detects drowsiness by    computing heart rate variation and using logistic regression based    machine learning algorithms.-   Driver Drowsiness Detection Based on Time Series Analysis of    Steering Wheel Angular Velocity (Zhenhai, DinhDat, Hongyu, Ziwen, &    Xinyu, 2017) uses a sliding window approach to analyze steering    wheel angular velocity and detect a state of drowsiness.-   Yawning detection by the analysis of variational descriptor for    monitoring driver drowsiness (Akrout & Mandi, 2016) performs    drowsiness detection by detecting yawning. Yawning is detected by    using the Viola-Jones method to detect the mouth and localizing the    external lips contour with the method of active contours.-   Real-time system for monitoring driver vigilance (Bergasa, Nuevo,    Sotelo, Barea, & Lopez, 2006) uses the features shown in section 1.0    to calculate drowsiness. They do not use convolutional neural    networks (CNNs) to detect faces or landmarks.

SUMMARY OF THE DISCLOSURE

According to a first aspect of the invention, there is provided adrowsiness detection system, which includes:

an image sensor, directed towards a face of an operator for capturing aseries of images of a face of the operator;

at least one movement sensor installed in a vehicle to monitor movementof the vehicle;

processing hardware which comprises a central processing unit, a datastorage facility in communication with the central processing unit andinput/output interfaces in communication with the central processingunit, the processing hardware being configured to implement a set oftrained convolution neural networks (CNNs) comprising:

a face detection group into which at least one image of the series ofimages is received from the image sensor for detecting the face of theoperator in the at least one image;

a face tracking group which match the face in the at least one image inthe series of images with the same face in a previous image in theseries of images and which then track the motion of the same face in thefurther series of images;

a facial landmark detection group which detects movement of a pluralityof facial landmarks in the further series of images, the faciallandmarks selected from the group of jawline, eyebrows, eyelids, nose,pupils and lips;

a face pose detection group which determine the orientation of the faceof the operator; and

a drowsiness detection group which includes any one or more of:

a yawning and talking detection group, which is trained to determinewhether an operator is talking, thereby to provide an indication ofalertness, and whether an operator is yawning, thereby to provide anindication of drowsiness;

a PERCLOS (percentage of eye closure over time) calculation, whichdetermines the percentage of eye closure over a period of time, therebyto provide an indication of drowsiness if the PERCLOS exceeds apredefined value;

a fixed gaze calculation, which determines when an operator's gaze timeexceeds a predefined value, thereby to provide an indication ofdrowsiness; and

an erratic movement detection group, which is trained to determine whena vehicle's movement resembles that of a drowsy driver, thereby toprovide an indication of drowsiness.

The drowsiness detection group may include an ensemble function, whichensembles outputs from any one or more of the yawning and talkingdetection group, the PERCLOS calculation, the fixed gaze calculation,and the erratic movement detection group thereby to generate an alarmwhen the ensembled value exceeds a predefined value for a drowsyoperator.

The drowsiness detection group may include a drowsiness ensemblecalculation which calculates the level of drowsiness of an operator byany one of:

predefined algorithms that calculate a numerical value representative ofthe drowsiness of the operator by using mathematical equations andconditional statements applied to the outputs from the yawning andtalking detection group, the PERCLOS detection group, the fixed gazedetection group and the erratic movement detection group; and

a machine learning algorithm trained with outputs from the yawning andtalking detection group, the PERCLOS detection group, the fixed gazedetection group and the erratic movement detection group to determine alevel of drowsiness of an operator.

The ensemble algorithm may include machine learning algorithms such asneural networks, regression tree ensembles, linear regression andpolynomial regression.

The drowsiness detection system may include an ambient light sensor,installed in proximity to the operator, in use to measure ambient lightin the environment of the operator to aid the machine learning andcomputer vision algorithms.

According to another aspect of the invention, there is provided amachine-implemented method for automated detection of drowsiness, whichincludes:

receiving onto processing hardware a series of images of an operator,processing the series of images of the operator on processing hardwareby a set of trained convolution neural networks comprising:

a face detection group into which at least one image of the series ofimages is received from the image sensor for detecting the face of theoperator in the at least one image;

a face tracking group which match the face in the at least one image inthe series of images with the same face in a previous image in theseries of images and which then track the motion of the same face in thefurther series of images;

a facial landmark detection group which detects movement of a pluralityof facial landmarks in the further series of images, the faciallandmarks selected from the group of jawline, eyebrows, eyelids, nose,pupils and lips;

a face pose detection group which determine the orientation of the faceof the operator; and

a drowsiness detection group, which includes any one or more of:

a yawning and talking detection group, which is trained to determinewhether an operator is talking, thereby to provide an indication ofalertness, and whether an operator is yawning, thereby to provide anindication of drowsiness;

a PERCLOS (percentage of eye closure over time) calculation, whichdetermines the percentage of eye closure over a period of time, therebyto provide an indication of drowsiness if the PERCLOS exceeds apredefined value;

a fixed gaze calculation, which determines when an operator's gaze timeexceeds a predefined value, thereby to provide an indication ofdrowsiness;

an erratic movement detection group, which is trained to determine whena vehicle movement exceeds a predefined value, thereby to provide anindication of drowsiness;

ensembling the outputs received from any one or more of the yawning andtalking detection group, the PERCLOS calculation, the a fixed gazecalculation, and the erratic movement detection group thereby togenerate an alarm when the ensembled value exceed a predefined value fora drowsy operator; and

generating an alarm when the level of drowsiness of the operatordetermined by the processing hardware exceeds a predefined value.

According to yet another aspect of the invention, there is provided amachine-implemented method for automated detection of drowsiness, whichincludes:

receiving from an imaging device directed at the face of an operator aseries of images of the face of the operator onto processing hardware;

on the processor detecting facial landmarks of the operator from theseries of images to determine the level of talking by the operator, thelevel of yawning of the operator, the PERCLOS of the operator;

on the processor detecting the facial pose of the operator from theseries of images to determine the level of gaze fixation by theoperator;

on the processor calculating the level of drowsiness of the operator byensembling the level of talking by the operator, the level of yawning ofthe operator, the PERCLOS of the operator and the level of gaze fixationby the operator; and

generating an alarm when the calculated level of drowsiness of theoperator exceeds a predefined value.

The step of detecting facial landmarks of the face of the operatorincludes the prior steps of:

detecting a face of the operator in the series of images; and

tracking the face of the operator in the series of images.

The step of detecting facial landmarks of the face of the operatorincludes detecting any one or more of:

movement of the jawline of the operator in the series of images;

movement of the eyebrows of the operator in the series of images;

movement of the eyelids of the operator in the series of images;

movement of the nose of the operator in the series of images;

movement of the pupils of the operator in the series of images; and

movement of the lips of the operator in the series of images.

The level of gaze fixation of an operator may be detected by calculatingthe variance of the operator's head orientation and the operator's pupilmovement, the variance of both the head's orientation and pupil movementbeing used to predict the drowsiness level of the operator. For example,an operator of a vehicle would normally be checking the rear-viewmirrors and side-view mirrors, blind spots and scanning the road ahead.As the drowsiness level of an operator increases, their gaze becomesfixated on a single area. The movement of their head and pupils,therefore, decrease significantly.

The invention is now described, by way of non-limiting example, withreference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings(s):

FIG. 1 shows a block diagram of a machine-implemented method forautomated detection of drowsiness in accordance with one aspect of theinvention;

FIG. 2 shows a diagram of the facial landmarks that are detected in themethod of FIG. 1; and

FIG. 3 shows a block diagram of a drowsiness detection system inaccordance with another aspect of the invention.

DETAILED DESCRIPTION

The drowsiness detection method makes use of multiple sensors, processesand algorithms that work together to output an accurate drowsinesslevel. An overview of the system is given in FIG. 1. The system startsby obtaining data from the sensors (110). The visual data is then passedto machine learning and computer vision algorithms (120) to be furtherprocessed. The processed data and data obtained from other sensors arethen passed to the second set of algorithms (130) that determine thelevel of drowsiness (139).

1. Sensors (110)

As shown in FIG. 1, the system makes use of several sensors (112, 114and 116) to provide data to the algorithms. The sensors used include butare not limited to ambient light sensors (112), imaging devices such ascameras (114) and gyroscopes and accelerometers (116).

The camera (114) may be placed anywhere provided that the face of theoperator is in view of the camera. A position that allows both eyes ofthe operator to be seen is beneficial since the method monitors the eyesto determine drowsiness. The system does not rely on the type of cameraused as long as it provides a clear image of the driver's face. Thismeans that the camera should have adequate resolution and shouldimplement a form of night vision if the system is required to operate atnight.

The ambient light sensor (112) should be placed close to the operator inorder to detect the amount of light in the environment.

The gyroscope (116) is attached to the steering wheel of the vehicle andmeasures the angular velocity of the steering wheel. The accelerometers(116) are placed at a fixed location on or in the vehicle and measuresthe sway (left/right sideways movement) of the vehicle. Either thegyroscope or the accelerometers (116), or both the gyroscope and theaccelerometer, may be used to detect erratic driving.

2. Machine Learning and Computer Vision Algorithms (120)

The machine learning and computer vision algorithms (120) receive visualdata from the camera (114) in the form of frames. They performprocessing on the frames in order to interpret their contents and outputnumeric data that can easily be used by other algorithms. Thesealgorithms make use of machine learning and CNNs to learn how tointerpret visual data. This is done in several steps. First, a large setof data must be collected on which the algorithm can train. Next, eachimage in the dataset needs to be labelled with the desired output of thenetwork. The network is then trained by supplying the image as an inputand comparing the output of the network with the desired output for theimage. If the output of the network matches the desired output, thenetwork does not need to update itself. If the output differs, thenetwork calculates an error and uses gradient descent to update theweights of the network to minimize the error. In this manner, thenetwork learns how to provide the desired output from an input image.

The first algorithm that is run is face detection (122). It takes animage as an input and outputs bounding boxes for all the faces detectedin the image. Once all of the faces have been detected, a face trackingalgorithm (124) is run. This uses the detected faces and tries to matchthem to the faces seen in the previous frame. It does this by performingface recognition on the faces and by tracking their motion.

Next, it performs facial landmark detection (126) on the detected faces.This once again takes the captured image as the input and outputs thepoints on the face which it has been trained to detect. The network canbe trained to detect points anywhere on the face. Since the goal is tomonitor the eyes and mouth, the network is trained to detect landmarksprimarily on the lips, eyelids and pupils of the face. FIG. 2 shows thefacial landmarks that are detected.

Lastly, a face pose detection network (128) is used to determine theorientation of the detected faces. The network takes a cropped face asthe input and outputs the orientation of the face.

As shown in FIG. 1, the machine learning algorithms also make use of anambient light sensor (112). This allows the algorithms to automaticallyadjust to the light conditions within the observed space, such as butnot limited to a cabin.

3. Drowsiness Detection Algorithms (130)

Several algorithms are implemented to calculate features that accuratelyclassify drowsiness. Their outputs are then ensembled (combined) todetermine whether the operator is drowsy.

The first algorithm determines whether the operator is yawning ortalking (132). Talking indicates that the operator is awake and mentallystimulated and therefore reduces the level of drowsiness. On the otherhand, yawning indicates that the operator is getting tired and thatdrowsiness may be present. The system detects both talking and yawningby using the output of the facial landmark detection algorithm (126). Itmonitors the distance between the points on the top and the bottom lipsof the operator. Talking is classified when the distance between thelips change rapidly for a period of time. Yawning is classified when thedistance increases and passed the normal open size of the mouth, thenremains open for a period of time and then closes again.

The second algorithm (PERCLOS) (134) monitors the eyelids of theoperator by analyzing the output of the facial landmark detectionalgorithm (126). It starts by creating a baseline for the operator'seyes by determining the nominal size of the operator's eyes when theyare open and closed. It then uses this baseline to calculate thepercentage of eye closure on each frame. The average percentage of eyeclosure (PERCLOS) is then calculated over two periods of time lastingtwo seconds and thirty seconds respectively. Since PERCLOS calculatesthe average eye closure over time, it inherently measures all of thecharacteristics that indicate drowsiness, namely, a smaller degree ofeyelid opening, slow eyelid movement, and an increased number of blinks.Furthermore, since it is calculated over both a short and long period oftime, it is able to detect instantaneous changes in drowsiness such asclosing of the eyes for an extended period of time and long-lastingchanges such as slowed blinking or a smaller degree of eye-opening.

The third algorithm performs fixed gaze detection (136) by monitoringthe pupils and the face pose (128). A vigilant operator is generallyexpected to look around often. In the case of a driver, he/she should bechecking the rear-view mirrors and side-view mirrors, blind spots andscanning the road ahead. As the drowsiness level of an operatorincreases, their gaze becomes fixated on a single area. The movement oftheir head and pupils, therefore, decrease significantly. The algorithmdetects this by calculating the variance of the head's orientation andthe pupil movement. The variance of both features is then used topredict the drowsiness level of the driver in the final algorithm. Ifthe function that the operator performs requires him/her to fix theirgaze, this feature will not be calculated.

Lastly, a specialized algorithm is implemented that detects drowsinessin drivers by detecting erratic driving (138) by monitoring the angularvelocity of the steering wheel (138) and/or the sway (left/rightsideways movement) of the vehicle. When drivers get drowsy they tend todrift out of their lane. When they realize that they have been driftingthey swerve back into their lane. This algorithm identifies the erraticdriving behavior by looking for short spikes in the angular velocityand/or acceleration and classifies it as drowsiness. If the operatorbeing monitored is not a driver, this feature will not be calculated.

The final algorithm (139) receives the drowsiness features output by theprevious algorithms as inputs. It can then calculate the finaldrowsiness level of the operator in two ways.

The first is by using equations or conditional statements that takes allthe features into account and outputs the level of drowsiness. Forexample, an equation can be implemented that simply weighs the outputsof all the features and outputs a percentage of drowsiness. Conditionalstatements can be added to improve performance, for instance, if thePERCLOS outputs for example 0.5 but the fixed gaze outputs for example0.1, the drowsiness output can be capped to for example 0.1 since theoperator is vigilant and looking around.

The second method makes use of machine learning algorithms. Thesealgorithms take the drowsiness features as inputs and are trained withmany examples of operators at different stages of drowsiness. It thenlearns the relationship between the features and the drowsiness level ofthe operator. Any machine learning algorithm or combination ofalgorithms may be used, this includes but is not limited to neuralnetworks, regression tree ensembles, linear regression and polynomialregression.

4. Hardware Implementation of the System 4.1 Portable Device (310)

The portable device gathers data from sensors (330) and runs algorithms(350), such as, but not limited to, machine learning and computer visionalgorithms (352) and drowsiness detection algorithms (354) on thecollected data.

The device interfaces with multiple sensors that may be built into thedevice or connected to it externally using either a wired or wirelessconnection. The type and number of sensors used will vary depending onthe nature of the algorithms that are running.

The sensors that may be used include, but are not limited to, cameras(332) (visible and infrared (IR)), global positioning system (GPS)(334), ambient light sensors (337), accelerometers (338), gyroscopes(336) and battery level sensors (339). The sensors may either send rawor processed data to the algorithms to be analysed.

Various algorithms (350) may be used to process and interpret the sensordata. The algorithms (350) may use data obtained directly from thesensors (330) and data obtained from the server (not shown). Thealgorithms (356) include a machine learning and computer visionalgorithm group (352) and a drowsiness detection group (354).

The portable device (310) is provided with a network interface (340)comprising of a Wi-Fi interface (342), a cellular network interface(GSM, HSPA, LTE) (344) and a Bluetooth interface (346) to communicatewith remote systems (not shown).

The portable device may also include a user interface (UI) (320) thatmay consist of a hardware user interface (HUI) (322) and/or a graphicaluser interface (GUI) (324). The UI can be used to log in to the system,control it and view information collected by it.

This system implements drowsiness detection by improving on existingmethods in two ways.

Firstly, it makes use of advanced machine learning and computer visionalgorithms to process a video stream and accurately extracts data fromit in real-time. This is done by training convolutional neural networks(CNNs) that take an image as an input and output the desired data.

Secondly, it uses the outputs of the machine learning and computervision algorithms to calculate several features that have been proven toaccurately classify drowsiness. These include PERCLOS, fixed gaze,yawning, talking and erratic driving detection when applied to drivers.An ensemble (combination) of these features is taken, together with datafrom other sensors, in order to accurately determine the drowsinesslevel of the operator.

The inventor is of the opinion that the invention as described providesa new and improved drowsiness detection system and a machine-implementedmethod for automated detection of drowsiness.

REFERENCES

-   [1] Australian Transport Council, “National Road Safety Strategy    2011-2020,” 2011.-   [2] Wikipedia, “Drunk driving law by country,” [Online]. Available:    https://en.wikipedia.org/wiki/Drunk_driving_law_by_country.-   [3] M. Babaeian, N. Bhardwaj, B. Esquivel and M. Mozumdar, “Real    time driver drowsiness detection using a logistic-regression-based    machine learning algorithm,” 2016 IEEE Green Energy and Systems    Conference (IGSEC), pp. 1-6, November 2016.-   [4] G. Zhenhai, L. DinhDat, H. Hongyu, Y. Ziwen and W. Xinyu,    “Driver Drowsiness Detection Based on Time Series Analysis of    Steering Wheel Angular Velocity,” 2017 9th International Conference    on Measuring Technology and Mechatronics Automation (ICMTMA), pp.    99-101, 2017.-   [5] B. Akrout and W. Mandi, “Yawning detection by the analysis of    variational descriptor for monitoring driver drowsiness,” 2016    International Image Processing, Applications and Systems (IPAS), pp.    1-5, 2016.-   [6] L. M. Bergasa, J. Nuevo, M. A. Sotelo, R. Barea and M. E. Lopez,    “Real-time system for monitoring driver vigilance,” IEEE    Transactions on Intelligent Transportation Systems, pp. 63-77, 2006.

1. A drowsiness detection system, which includes: an image sensor,directed towards a face of an operator for capturing a series of imagesof a face of the operator; at least one movement sensor installed in avehicle to monitor movement of the vehicle; processing hardware whichcomprises a central processing unit, a data storage facility incommunication with the central processing unit and input/outputinterfaces in communication with the central processing unit, theprocessing hardware being configured to implement a set of trainedconvolution neural networks (CNNs) comprising: a face detection groupinto which at least one image of the series of images is received fromthe image sensor for detecting the face of the operator in the at leastone image; a face tracking group which match the face in the at leastone image in the series of images with the same face in a previous imagein the series of images and which then track the motion of the same facein the further series of images; a facial landmark detection group whichdetects movement of a plurality of facial landmarks in the furtherseries of images, the facial landmarks selected from the group ofjawline, eyebrows, eyelids, nose, pupils and lips; a face pose detectiongroup which determine the orientation of the face of the operator; and adrowsiness detection group which includes any one or more of: a yawningand talking detection group, which is trained to determine whether anoperator is talking, thereby to provide an indication of alertness, andwhether an operator is yawning, thereby to provide an indication ofdrowsiness; a PERCLOS (percentage of eye closure over time) calculation,which determines the percentage of eye closure over a period of time,thereby to provide an indication of drowsiness if the PERCLOS exceeds apredefined value; a fixed gaze calculation, which determines when anoperator's gaze time exceeds a predefined value, thereby to provide anindication of drowsiness; an erratic movement detection group, which istrained to determine when a vehicle movement exceeds a predefined value,thereby to provide an indication of drowsiness.
 2. The drowsinessdetection system of claim 1, in which the drowsiness detection groupincludes an ensemble algorithm, which ensembles outputs from any one ormore of the yawning and talking detection group, the PERCLOScalculation, the fixed gaze calculation, and the erratic movementdetection group thereby to determine the drowsiness of an operator. 3.The drowsiness detection system of claim 2, in which the ensemblealgorithm calculates the level of drowsiness of an operator by any oneof: predefined algorithms that calculate a numerical valuerepresentative of the drowsiness of the operator by using mathematicalequations and conditional statements applied to the outputs from theyawning and talking detection group, the PERCLOS detection group, thefixed gaze detection group and the erratic movement detection group; anda machine learning algorithm trained with outputs from the yawning andtalking detection group, the PERCLOS detection group, the fixed gazedetection group and the erratic movement detection group to determine alevel of drowsiness of an operator.
 4. The drowsiness detection systemof claim 3, in which the ensemble algorithm includes machine learningalgorithms selected from any one or more of neural networks, regressiontree ensembles, linear regression and polynomial regression.
 5. Thedrowsiness detection system of claim 1, which includes an ambient lightsensor, installed in proximity to the operator, in use to measureambient light in the environment of the operator to aid the machinelearning and computer vision algorithms.
 6. A machine-implemented methodfor automated detection of drowsiness, which includes: receiving ontoprocessing hardware a series of images of an operator, processing theseries of images of the operator on processing hardware by a set oftrained convolution neural networks comprising: a face detection groupinto which at least one image of the series of images is received fromthe image sensor for detecting the face of the operator in the at leastone image; a face tracking group which match the face in the at leastone image in the series of images with the same face in a previous imagein the series of images and which then track the motion of the same facein the further series of images; a facial landmark detection group whichdetects movement of a plurality of facial landmarks in the furtherseries of images, the facial landmarks selected from the group ofjawline, eyebrows, eyelids, nose, pupils and lips; a face pose detectiongroup which determine the orientation of the face of the operator; and adrowsiness detection group, which includes any one or more of: a yawningand talking detection group, which is trained to determine whether anoperator is talking, thereby to provide an indication of alertness, andwhether an operator is yawning, thereby to provide an indication ofdrowsiness; a PERCLOS (percentage of eye closure over time) calculation,which determines the percentage of eye closure over a period of time,thereby to provide an indication of drowsiness if the PERCLOS exceeds apredefined value; a fixed gaze calculation, which determines when anoperator's gaze time exceeds a predefined value, thereby to provide anindication of drowsiness; an erratic movement detection group, which istrained to determine when a vehicle's movement resembles that of adrowsy driver, thereby to provide an indication of drowsiness;ensembling the outputs received from any one or more of the yawning andtalking detection group, the PERCLOS calculation, the a fixed gazecalculation, and the erratic movement detection group thereby togenerate an alarm when the ensembled value exceed a predefined value fora drowsy operator; and generating an alarm when the level of drowsinessof the operator determined by the processing hardware exceeds apredefined value.
 7. A machine-implemented method for automateddetection of drowsiness, which includes receiving from an imaging devicedirected at the face of an operator a series of images of the face ofthe operator onto processing hardware, on the processor detecting faciallandmarks of the operator from the series of images to determine thelevel of talking by the operator, the level of yawning of the operator,the PERCLOS of the operator; on the processor detecting the facial poseof the operator from the series of images to determine the level of gazefixation by the operator; on the processor calculating the level ofdrowsiness of the operator by ensembling the level of talking by theoperator, the level of yawning of the operator, the PERCLOS of theoperator and the level of gaze fixation by the operator; and generatingan alarm when the calculated level of drowsiness of the operator exceedsa predefined value.
 8. The machine-implemented method for automateddetection of drowsiness as claimed in claim 7, in which the step ofdetecting facial landmarks of the face of the operator includes theprior steps of: detecting a face of the operator in the series ofimages; and tracking the face of the operator in the series of images.9. The machine-implemented method for automated detection of drowsinessas claimed in claim 8, in which the step of detecting facial landmarksof the face of the operator includes detecting any one or more of:movement of the jawline of the operator in the series of images;movement of the eyebrows of the operator in the series of images;movement of the eyelids of the operator in the series of images;movement of the nose of the operator in the series of images; movementof the pupils of the operator in the series of images; and movement ofthe lips of the operator in the series of images.
 10. Themachine-implemented method for automated detection of drowsiness asclaimed in claim 7, in which the level of gaze fixation of an operatoris detected by calculating the variance of the operator's headorientation and the operator's pupil movement, the variance of both thehead's orientation and pupil movement being used to predict thedrowsiness level of the operator.